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A general interest in attrition, or loss of units 
from a study, stems in part from the observation that the infreguency 
of attention to attrition exacerbates problems of data 
interpretation. As a substudy of the national evaluation of Project 
Follow Through, the potential biasing effects of attrition of 
subjects from the sites were investigated. Policy attrition, the 
administrative dropping of a unit; program attrition, loss of 
subjects due to mobility, dislike of the treatment, etc.; and sample 
attrition, units omitted because of deficient data, have been 
defined, but were not distinguished in this research. The 
relationship between attrition, pretest score, and income level were 
studied. In the few cases where additional variance was accounted for 
by the interaction of pretest or income with treatment within-site, 
an interpretation of the possible biasing effects of attrition was 
made in the evaluation of the specific site. It is suggested that in 
cases where differential attrition is found, the offending variable 
be considered for use as a covariate. (BW) 
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Introduction 

One of the most important problems that is germane to the topic of 
social experimentation, educational evaluations, and especially to longi- 
tudinal studies is the issue of attrition, that is, the loss of units 
from a study. In a critique of educational evaluation studies including 
Sesame Street, Head Start, and Follow Through, Anderson (1973) suggested 
that additional information as to the nature of these programs be 
provided to assist the reader in interpreting the major conclusions. 
Such information, which has largely been omitted in the past, should 
include "... provision of data on dropouts in groups of subjects. 
(Pailtire to take account of differences in the number and kinds of 
dropouts in groups that are to be compared reprerents a major source of 
error in conclusions about the effects of educational treatments.)" 
(Anderson, 1973, p. 202). 

While the significance of attrition problems was popularized by 
Campbell and Stanley (1963) in their identification of attrition as a 
"possible rival hypothesis," later work by Jurs and Glass (1970) 
revealed that mortaility estimates are seldom given in reports cf educational 
studies, and that virtually no textbooks include attrition as a topic of 
interest. Although Kershaw (1971) gives suggestions for countering attri- 
tion in interview situations, and Reicken and Boruch (1975) as well as 
Jurs and Glass (1970) discuss various aspects of the attrition problem, 
it is clear from a review of educational studies that investigations of 
the effects of attrition on either the internal or external validity of 
educational studies are usually lacking. 

Our general interest in attrition stems in part from the observa- 
tion that the infrequency of attention to attrition exacerbates problems 
of data interpretation. We are concerned that the lack of a clear 
understanding of the causes and effects of attrition will inhibit the 
development of better evaluation technology and will obscure the 
identification of valuable educational and social programs. 

In order to give the present paper a real world referent, we will 
orient the rest of the discussion around our experience with a study of 
attrition in the national evaluation of Follow Through, a large-scale 
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quasi -experimental program in compensatory education. Follow Through 
was implemented through an approach known as "planned variation" which 
included the systematic introduction of a variety of programs into the 
kindergarten through third grade years of public education by educational 
specialists (sponsors) from research i nstitutions and universities who 
each implemented their own educational model in a group of school dis- 
tricts (sites) • 

The national evaluation of Follow Through, in which Abt Associates 
Inc. (AAI) has participated since July, 1972, is designed to examine the 
effects of different approaches to education for improving the perform- 
ance of disadvantaged children in a variety of areas. The national 
evaluation has concentrated on examining data collected from groups of 
program participants (Cohorts) in each year since X969. These data in- 
clude tests administered to children, questionnaires submitted by teachers, 
and parent interviews. As a substudy, AAI has investigated whether attri- 
tion of subjects from the sites included in the national evaluatie ,1 of 
Cohort II (subjects who entered Follow Through in fall, 1970) has biased 
that evaluation. Many of the ideas incorporated in the present paper 
originated in that study of attrition. 

The "analytic sample" for the above-referenced Follow Through 
evaluation included a total of 5,519 children (3,369 treatment and 2,152 
comparison) distributed across 17 sponsors, where each sponsor implemented 
its educational program in between one and seven sites and where each site 
contained a Follow Through treatment group (FT) and a non-Follow Through 
comparison group (NFT) . 

General Attrition Concerns 

Before addressing the problems of identifying and explaining 
attrition in Follow Through, a discussion of some specific ways in which 
attrition might or might not have biesed the Follow Through (or any other) 
evaluation nay be helpful. Assume that the design discussed above holds, 
and that the "analytic sample" of children has been identified, and is 
smaller — due to attrition — than the initial evaluation sample. If 
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subjects who have not been retained in the analytic sample are different 
in some '•important 11 way from those who have been retained, then the 
internal and external validity of the study may be called to question. 
To pursue this matter further, we ~iay define three general cases. First, 
if dropouts in both the treatment and comparison groups are similar to 
each other, and to those who remained, then attrition is not a threat 
to the study. On the other hand, if the dropouts from both the treatment 
and comparison groups are similar to each other but are not representative 
of those who remain, the external validity of the study is weakened. 
Finally, if the treatment and comparison dropouts are not representative 
of the remaining subjects, and in addition arc not similar to each other, 
both the internal and external validity of the study is questionable. 

Stated in terms of jhe Follow Through evaluation, if in both the 
treatment and control groups low ability children have been retained 
while high ability children dropped out, our results would be interpretable 
in terms of low ability children, but we would not know the effects of 
the program on high ability children, i.e., our ability to generalize to 
the population represented by the entry group would be diminished. This 
does not mean that we cannot analyze the data on the retained sample, 
it is to say that the interpretability of such a primary analysis is 
limited. Turning to another example, if we were to fi~#l that low ability 
children dropped ort of the treatment group, while high ability children 
dropped out of the comparison group, then the possibility of making valid 
statements as to the effectiveness of the treatment is limited. Again, 
this is not to say that the data cannot or should not be analyzed, rather 
that we must use the results of such an attrition finding to understand 
and perhaps qualify the results of the primary analysis. 

Definition of Attrition 

Although commonly thought of as the loss of subjects from an 
experiment due to mobility, death, dislike of the treatment, etc., the 
phenomenon of attrition may be defined in a number of ways. The definition 
developed for use in the Follow Through evaluation is a good deal more 
comprehensive than most, and consists of three major categories: Policy, 
Program and Sample. 
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Policy attrition is defined as the administrative dropping of a 
unit (at any level) after the initial implementation of a program. Within 
the Follow Through evaluation, this might mean the permanent cessation of 
data collection in an entire site, or within a single school in that 
site. Policy attrition may even occur at the child level if for instance 
the policy of a gv-en school is to return all children whose academic 
performance has improved to a "regular" classroom in order to provide 
space for the more needy child. 

Program attrition defines the types c subject behaviors that are 
generally thought of when attrition is mentioned. This includes the loss 
of subjects due to mobility, dislike of the treatment, illness or death, 
etc. It is difficult if r,~ u impossible at tl ~hild level to differentiate 
between policy and program attrition unless daca have been specifically 
collected for that purpose. For example, it may not be possible to 
differentiate between a child being dropped out of the program because 
his academic performance improved and is no longer in need of the 
treatment, and the child who is dropped because his parents left town. 

Sample attrition occurs within the office of the evaluator and/or 
data collector. Included in this category are subjects who are omitted 
from the analytic sample for any number of data deficiency reasons 
including an invalid test, a large amount of missing data, or inadequate 
cell size. 

While we have conceptualized attrition as being of three major 
types, this year we have not distinguished between different types of 
attrition in our analyses. This was done because our primary concern 
has been to ascertain whether the analytic sample was representative of 
the sample which was present at the beginning of the program. In order 
to answer this question it was necessary to collapse across our 
attrition definitions. (See Figure 1.) 

Analytic Model 

This year within the Follow Through evaluation we chose to 
examine attrition by exploring its relationship with two measures - a 
pretest score and an income index - both of which were obtained during 
the first year of a child's participation in Follow Through. These two 
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variables capture much of the essence of a host of available background 
measures and in addition are included as covariates in our primary analysis 
(an analysis of covariance model which compares FT and NFT posttest scores) . 

The data were analyzed in order to answer the following question: 
for a given sponsor, does the FT/NPT within-site difference in rate of 
attrition differ across levels of pretest or income? A hierarchical 
regression analysis was run for each sponsor to predict a dichotomous 
dependent variable (child was in the analytic sample vs. child was not 
in the analytic sample). The regression model was as follows: 

Y - a o u + b i x i + » 2 x 2 + ... + b A ) + (b i+1 x. +1 ♦ ... + bjXj ) 

+ ^j+iVi + ... + b k x k ) + ( b k+1 x k+1 + _ + w 

Where set A is composed of variables K ± through X^ set B is composed of 
variables X k+1 through X L , and variables are defined as follows: 

Y = dichotomous dependent variable 

1 = not in the analytic sample 

0 = in the analytic sample 
a^ = constant 
U = unit vector 

b. - . . b = regression weights for X, ... X 
a 1j 1 L 

X^ = pretest Cor incoice) 

X 2 X i m (number of sites - 1) orthogonal between-sites 
Helmert contrasts 

X i+1 * * * X i = ( numl3er °f sites* ) orthogonally coded treatment 
within-site contrasts 

.5 = FT 
-.5 * NFT 

X j+1 *** *k = interacfcion of X 1 and X 2 ... X i 
X k+1 * # * X L = intera ction of X x and X ±+1 . . . x 
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An P test was performed to determine the significance of the 
incremental variance added by set B, the last set of coefficients in 
the model (b k+1 b L : the interaction of pretest with treatment 
within-site). if the overall F test proved non-significant, that is, if 
the set of interaction variables failed to add "significantly" to R 2 
we proceeded no further. If, however, the interaction terms (set B) 
explained variance in the dependent variable, we examined the individual 
treatment within-site coefficients. 

The major question we asked in our attrition study in Follow 
Through was whether the FT/NFT within-site difference in rate of 
attrition was different across levels of pretest and income. The answer, 
in most cases, was no. In the few cases where additional variance was 
accounted f^r by the interaction of pretest or income with treatment 
within-site an interpretation of the possible biasing effects of attrition 
was made in the evaluation of the specific site. For example, Figure 2 
displays two hypothetical within-site regression lines of attrition 
rate on pretest. In such a site the attrition rate within FT is highest 
for lower scoring children and lowest for high scoring children, while 
the NFT rate of attrition is relatively constant across levels of the 
pretest. 

FIGURE 2 

Regression Lines of Attrition Rate on Pretest 
for FT and NFT Groups in a Hypothetical Site 
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We suggest that if one does find differential attrition tha* the 
offending variable be considered for u;se as a covariate. Assuming that 
the range of the variable has not been shifted and that the treatment 
and comparison groups have similar distributions on the variable, cov^riance 
will work toward the proper adjustment of the variable. 

It is apparent that a full treatment of attrition requires that 
two types of data be collected. First/ data on potential biasing variables 
must be available for both the dropouts and the remaining subjects. This 
implies that such data should be gathered as early as possible in the 
design/implementation of a program. Second, special data as to the nature 
of attrition and the reasons that a subject dropped out of a program must 
be collected if one is interested in exploring differences between 
different types of dropouts. It is our recommendation that such data be 
collected in all social science studies where attrition is likely to occur. 
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